我们在室外环境中自动驾驶的背景下研究了视觉和语言导航(VLN)问题。我们通过明确接地与Textual命令相对应的可通道区域来解决问题。在每个时间戳,该模型预测与中间或最终可通道区域相对应的分割掩码。我们的工作与VLN中的现有工作形成鲜明对比,VLN的现有工作将该任务置于节点选择问题,并且给定与环境相对应的离散连接图。我们不假定这种离散的地图的可用性。我们的工作朝着动作领域的连续性发展,通过视觉反馈提供了解释性,并允许在需要更精细的操作的命令上进行VLN,例如“两辆汽车之间的停车”。此外,我们提出了一种新型的元数据carla-nav,以允许有效的训练和验证。该数据集包括预录制的培训序列以及用于验证和测试的实时环境。我们提供广泛的定性和定量经验结果,以验证所提出的方法的功效。
translated by 谷歌翻译
人类有自然能够毫不费力地理解语言指挥,如“黄色轿车旁边的公园”,本能地知道车辆的道路的哪个地区应该导航。扩大这种对自主车辆的能力是创建根据人类命令响应和行动的完全自治代理的下一步。为此,我们提出了通过语言命令引用可导航区域(RNR),即导航的接地区域的新任务。 RNR与引用图像分割(RIS)不同,该图像分割(RIS)侧重于自然语言表达式而不是接地导航区域的对象接地。例如,对于指令“黄色轿车旁边的公园,”RIS将旨在分割推荐的轿车,而RNR旨在将建议的停车位分段在道路上分割。我们介绍了一个新的DataSet,talk2car-regseg,它将现有的talk2car数据集扩展,其中包含语言命令描述的区域的分段掩码。提供了一个单独的测试拆分,具有简明的机动指导命令,以评估我们数据集的实用性。我们使用新颖的变换器的架构基准测试所提出的数据集。我们呈现广泛的消融,并在多个评估指标上显示出卓越的性能。基于RNR输出产生轨迹的下游路径规划器确认了所提出的框架的功效。
translated by 谷歌翻译
多视图检测(MVD)对于拥挤环境中的遮挡推理非常有效。虽然最近使用深度学习的作品在该领域取得了重大进展,但它们已经忽略了泛化方面,这使得它们\ emph {现实世界部署不切实际。我们工作的关键新颖性是\ emph {形式化}三种临界形式的普遍化和\ emph {建议实验来评估它们}:泛化与i)不同数量的相机,ii)变化的相机位置,最后,iii)到新场景。我们发现现有的最先进的模型通过对单个场景和相机配置过度提供了较差的概括。为了解决问题:(a)我们提出了一种新颖的通用MVD(GMVD)数据集,同时使用变化的日间,相机配置,不同数量的相机以及(B)来吸收多样化的场景,以及(B)我们讨论了对MVD带来概括的属性并提出一个鞍座模型融合它们。我们在WildTrack,MultiviewX和GMVD数据集上执行一套全面的实验,以激励评估MVD方法的概括能力,并证明所提出的方法的功效。可以在\ url {https:github.com/jeetv/gmvd}中找到代码和建议的数据集
translated by 谷歌翻译
我们研究了参考图像分割(RIS),该图像分割(RIS)输出与自然语言描述相对应的分割图。有效地解决RIS需要考虑发生\ emph {跨}视觉和语言模态以及每种模态的交互。现有方法受到限制,因为它们要么计算不同形式的交互作用\ emph {secentally}(导致错误传播)或\ emph {nighore}。我们通过通过同步多模式融合模块(SFM)执行所有三个交互\ emph {同时}来解决此限制。此外,为了产生精致的分割面膜,我们提出了一种新型的层次交叉模式聚合模块(HCAM),其中语言特征有助于在整个视觉层次结构上交换上下文信息。我们介绍了彻底的消融研究,并在四个基准数据集上验证方法的性能,显示出对现有最新方法(SOTA)方法的性能增长。
translated by 谷歌翻译
Multiple studies have focused on predicting the prospective popularity of an online document as a whole, without paying attention to the contributions of its individual parts. We introduce the task of proactively forecasting popularities of sentences within online news documents solely utilizing their natural language content. We model sentence-specific popularity forecasting as a sequence regression task. For training our models, we curate InfoPop, the first dataset containing popularity labels for over 1.7 million sentences from over 50,000 online news documents. To the best of our knowledge, this is the first dataset automatically created using streams of incoming search engine queries to generate sentence-level popularity annotations. We propose a novel transfer learning approach involving sentence salience prediction as an auxiliary task. Our proposed technique coupled with a BERT-based neural model exceeds nDCG values of 0.8 for proactive sentence-specific popularity forecasting. Notably, our study presents a non-trivial takeaway: though popularity and salience are different concepts, transfer learning from salience prediction enhances popularity forecasting. We release InfoPop and make our code publicly available: https://github.com/sayarghoshroy/InfoPopularity
translated by 谷歌翻译
Speech systems are sensitive to accent variations. This is especially challenging in the Indian context, with an abundance of languages but a dearth of linguistic studies characterising pronunciation variations. The growing number of L2 English speakers in India reinforces the need to study accents and L1-L2 interactions. We investigate the accents of Indian English (IE) speakers and report in detail our observations, both specific and common to all regions. In particular, we observe the phonemic variations and phonotactics occurring in the speakers' native languages and apply this to their English pronunciations. We demonstrate the influence of 18 Indian languages on IE by comparing the native language pronunciations with IE pronunciations obtained jointly from existing literature studies and phonetically annotated speech of 80 speakers. Consequently, we are able to validate the intuitions of Indian language influences on IE pronunciations by justifying pronunciation rules from the perspective of Indian language phonology. We obtain a comprehensive description in terms of universal and region-specific characteristics of IE, which facilitates accent conversion and adaptation of existing ASR and TTS systems to different Indian accents.
translated by 谷歌翻译
In molecular research, simulation \& design of molecules are key areas with significant implications for drug development, material science, and other fields. Current classical computational power falls inadequate to simulate any more than small molecules, let alone protein chains on hundreds of peptide. Therefore these experiment are done physically in wet-lab, but it takes a lot of time \& not possible to examine every molecule due to the size of the search area, tens of billions of dollars are spent every year in these research experiments. Molecule simulation \& design has lately advanced significantly by machine learning models, A fresh perspective on the issue of chemical synthesis is provided by deep generative models for graph-structured data. By optimising differentiable models that produce molecular graphs directly, it is feasible to avoid costly search techniques in the discrete and huge space of chemical structures. But these models also suffer from computational limitations when dimensions become huge and consume huge amount of resources. Quantum Generative machine learning in recent years have shown some empirical results promising significant advantages over classical counterparts.
translated by 谷歌翻译
Developing and least developed countries face the dire challenge of ensuring that each child in their country receives required doses of vaccination, adequate nutrition and proper medication. International agencies such as UNICEF, WHO and WFP, among other organizations, strive to find innovative solutions to determine which child has received the benefits and which have not. Biometric recognition systems have been sought out to help solve this problem. To that end, this report establishes a baseline accuracy of a commercial contactless palmprint recognition system that may be deployed for recognizing children in the age group of one to five years old. On a database of contactless palmprint images of one thousand unique palms from 500 children, we establish SOTA authentication accuracy of 90.85% @ FAR of 0.01%, rank-1 identification accuracy of 99.0% (closed set), and FPIR=0.01 @ FNIR=0.3 for open-set identification using PalmMobile SDK from Armatura.
translated by 谷歌翻译
Selective classification involves identifying the subset of test samples that a model can classify with high accuracy, and is important for applications such as automated medical diagnosis. We argue that this capability of identifying uncertain samples is valuable for training classifiers as well, with the aim of building more accurate classifiers. We unify these dual roles by training a single auxiliary meta-network to output an importance weight as a function of the instance. This measure is used at train time to reweight training data, and at test-time to rank test instances for selective classification. A second, key component of our proposal is the meta-objective of minimizing dropout variance (the variance of classifier output when subjected to random weight dropout) for training the metanetwork. We train the classifier together with its metanetwork using a nested objective of minimizing classifier loss on training data and meta-loss on a separate meta-training dataset. We outperform current state-of-the-art on selective classification by substantial margins--for instance, upto 1.9% AUC and 2% accuracy on a real-world diabetic retinopathy dataset. Finally, our meta-learning framework extends naturally to unsupervised domain adaptation, given our unsupervised variance minimization meta-objective. We show cumulative absolute gains of 3.4% / 3.3% accuracy and AUC over the other baselines in domain shift settings on the Retinopathy dataset using unsupervised domain adaptation.
translated by 谷歌翻译
Many real-world learning scenarios face the challenge of slow concept drift, where data distributions change gradually over time. In this setting, we pose the problem of learning temporally sensitive importance weights for training data, in order to optimize predictive accuracy. We propose a class of temporal reweighting functions that can capture multiple timescales of change in the data, as well as instance-specific characteristics. We formulate a bi-level optimization criterion, and an associated meta-learning algorithm, by which these weights can be learned. In particular, our formulation trains an auxiliary network to output weights as a function of training instances, thereby compactly representing the instance weights. We validate our temporal reweighting scheme on a large real-world dataset of 39M images spread over a 9 year period. Our extensive experiments demonstrate the necessity of instance-based temporal reweighting in the dataset, and achieve significant improvements to classical batch-learning approaches. Further, our proposal easily generalizes to a streaming setting and shows significant gains compared to recent continual learning methods.
translated by 谷歌翻译